Repository navigation
feat: dlopen Enzyme - #149271
feat: dlopen Enzyme#149271
Conversation
|
These commits modify the If this was unintentional then you should revert the changes before this PR is merged. Some changes occurred in compiler/rustc_codegen_llvm/src/llvm/enzyme_ffi.rs cc @ZuseZ4 |
|
That's exciting, and is gonna make some people on the infra team very happy. Just to be sure, are the tests still passing? |
This comment has been minimized.
This comment has been minimized.
|
(oops, sorry I should have run tests before opening this pr. It takes too much time running tests on my m1 mac so let me check them tomorrow, including ci failures..... 🙇) |
|
Don't worry. I don't run all tests either before opening a PR either, that indeed takes way too long. I mostly run |
|
Oh sorry, I was only referring to all the autodiff tests mentioned here. I'm also not running all tests. |
This comment has been minimized.
This comment has been minimized.
|
@sgasho It seems that all autodiff tests invoking enzyme fail, e.g: I still remember I had the same issue in my original PR, so that might be the reason. |
|
Some changes occurred in compiler/rustc_codegen_llvm/src/builder/autodiff.rs cc @ZuseZ4 |
I solved the "EnzymeWrapper not initialized" problem in the latest commit (52f0d2e) However, another error occurred e.g. I could not figure out what triggers "This analysis pass was not registered prior to being queried". codex cli says Enzyme has a problem but I'm not sure if it's right. I'll take look into it tomorrow.
|
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
|
Do you have a backtrace? |
|
I saw no further information using RUST_BACKTRACE=1 or |
|
You will have to use a debugger. These are LLVM aborts, not rust panics, so |
|
I see! |
|
Good that you properly solved the first bug. I had just commented it out back in my PR and then ended up running into the same bug you're seeing now as well |
|
To be more clear at what did (not) work. In my first attempt where I just dlopen'ed Enzyme directly in |
I tried removing the Mutex and running the test, whose result didn't change. I'm not sure but I think the cause may lie in Enzyme commits made from early October until this PR was submitted. I realized that DominatorTreeAnalysis and LoopAnalysis is not registered when TypeAnalyzer::TypeAnalyzer tries to access them. I added a code to register those passes in PreProcessCache::PreProcessCache(enzyme/Enzyme/FunctionUtils.cpp) like below and the "This analysis pass was not registered prior to being queried" error disappeared
|
this error has been resolved, but I encountered another error in abi_handling.rs. Details
|
I'm not sure if this is the best way to solve the problem... |
87537c4 to
58aeab5
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
|
I confirmed that all of the autodiff-related tests have passed same as #149271 (comment) (I fixed some llvm ir checks though) |
|
@bors r+ |
feat: dlopen Enzyme related issue: rust-lang#145899 related pr: rust-lang#146623 This PR is a continuation of rust-lang#146623 I refactored some code for rust-lang#146623 and added the functions shown in rust-lang#144197 r? `@bjorn3` cc: `@ZuseZ4` Zulip link: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libload.20.2F.20dlopen.20Enzyme.2Fautodiff/near/553647912
Rollup of 9 pull requests Successful merges: - #148756 (Warn on codegen attributes on required trait methods) - #148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - #149271 (feat: dlopen Enzyme) - #149354 (Bootstrap config: libgccjit libs dir) - #149459 (std: sys: fs: uefi: Implement set_times and set_perm) - #149950 (Simplify how inline asm handles `MaybeUninit`) - #150000 (Port `#[rustc_legacy_const_generics]` to use attribute parser ) - #150014 (Metadata loader cleanups) - #150021 (document that mpmc channels deliver an item to (at most) one receiver) r? `@ghost` `@rustbot` modify labels: rollup
feat: dlopen Enzyme related issue: rust-lang#145899 related pr: rust-lang#146623 This PR is a continuation of rust-lang#146623 I refactored some code for rust-lang#146623 and added the functions shown in rust-lang#144197 r? ``@bjorn3`` cc: ``@ZuseZ4`` Zulip link: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libload.20.2F.20dlopen.20Enzyme.2Fautodiff/near/553647912
Rollup of 8 pull requests Successful merges: - #148756 (Warn on codegen attributes on required trait methods) - #148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - #149271 (feat: dlopen Enzyme) - #149459 (std: sys: fs: uefi: Implement set_times and set_perm) - #149950 (Simplify how inline asm handles `MaybeUninit`) - #150000 (Port `#[rustc_legacy_const_generics]` to use attribute parser ) - #150014 (Metadata loader cleanups) - #150021 (document that mpmc channels deliver an item to (at most) one receiver) r? `@ghost` `@rustbot` modify labels: rollup
feat: dlopen Enzyme related issue: rust-lang#145899 related pr: rust-lang#146623 This PR is a continuation of rust-lang#146623 I refactored some code for rust-lang#146623 and added the functions shown in rust-lang#144197 r? ```@bjorn3``` cc: ```@ZuseZ4``` Zulip link: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libload.20.2F.20dlopen.20Enzyme.2Fautodiff/near/553647912
Rollup of 14 pull requests Successful merges: - #148756 (Warn on codegen attributes on required trait methods) - #148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - #149271 (feat: dlopen Enzyme) - #149459 (std: sys: fs: uefi: Implement set_times and set_perm) - #149771 (bootstrap readme: make easy to read when editor wrapping is not enabled) - #149856 (Provide an extended framework for type visit, for use in rust-analyzer) - #149950 (Simplify how inline asm handles `MaybeUninit`) - #150014 (Metadata loader cleanups) - #150021 (document that mpmc channels deliver an item to (at most) one receiver) - #150022 (Generate macro expansion for rust compiler crates docs) - #150029 (Update books) - #150031 (assert impossible branch is impossible) - #150034 (do not add `I-prioritize` when `F-*` labels are present) - #150036 (Use the embeddable filename for coverage artifacts) r? `@ghost` `@rustbot` modify labels: rollup
Rollup of 13 pull requests Successful merges: - #148756 (Warn on codegen attributes on required trait methods) - #148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - #149271 (feat: dlopen Enzyme) - #149459 (std: sys: fs: uefi: Implement set_times and set_perm) - #149771 (bootstrap readme: make easy to read when editor wrapping is not enabled) - #149856 (Provide an extended framework for type visit, for use in rust-analyzer) - #149950 (Simplify how inline asm handles `MaybeUninit`) - #150014 (Metadata loader cleanups) - #150021 (document that mpmc channels deliver an item to (at most) one receiver) - #150029 (Update books) - #150031 (assert impossible branch is impossible) - #150034 (do not add `I-prioritize` when `F-*` labels are present) - #150036 (Use the embeddable filename for coverage artifacts) r? `@ghost` `@rustbot` modify labels: rollup
Rollup merge of #149271 - sgasho:enzyme-dlopen, r=bjorn3 feat: dlopen Enzyme related issue: #145899 related pr: #146623 This PR is a continuation of #146623 I refactored some code for #146623 and added the functions shown in #144197 r? ````@bjorn3```` cc: ````@ZuseZ4```` Zulip link: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libload.20.2F.20dlopen.20Enzyme.2Fautodiff/near/553647912
Rollup of 13 pull requests Successful merges: - rust-lang/rust#148756 (Warn on codegen attributes on required trait methods) - rust-lang/rust#148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - rust-lang/rust#149271 (feat: dlopen Enzyme) - rust-lang/rust#149459 (std: sys: fs: uefi: Implement set_times and set_perm) - rust-lang/rust#149771 (bootstrap readme: make easy to read when editor wrapping is not enabled) - rust-lang/rust#149856 (Provide an extended framework for type visit, for use in rust-analyzer) - rust-lang/rust#149950 (Simplify how inline asm handles `MaybeUninit`) - rust-lang/rust#150014 (Metadata loader cleanups) - rust-lang/rust#150021 (document that mpmc channels deliver an item to (at most) one receiver) - rust-lang/rust#150029 (Update books) - rust-lang/rust#150031 (assert impossible branch is impossible) - rust-lang/rust#150034 (do not add `I-prioritize` when `F-*` labels are present) - rust-lang/rust#150036 (Use the embeddable filename for coverage artifacts) r? `@ghost` `@rustbot` modify labels: rollup
Rollup of 13 pull requests Successful merges: - rust-lang/rust#148756 (Warn on codegen attributes on required trait methods) - rust-lang/rust#148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - rust-lang/rust#149271 (feat: dlopen Enzyme) - rust-lang/rust#149459 (std: sys: fs: uefi: Implement set_times and set_perm) - rust-lang/rust#149771 (bootstrap readme: make easy to read when editor wrapping is not enabled) - rust-lang/rust#149856 (Provide an extended framework for type visit, for use in rust-analyzer) - rust-lang/rust#149950 (Simplify how inline asm handles `MaybeUninit`) - rust-lang/rust#150014 (Metadata loader cleanups) - rust-lang/rust#150021 (document that mpmc channels deliver an item to (at most) one receiver) - rust-lang/rust#150029 (Update books) - rust-lang/rust#150031 (assert impossible branch is impossible) - rust-lang/rust#150034 (do not add `I-prioritize` when `F-*` labels are present) - rust-lang/rust#150036 (Use the embeddable filename for coverage artifacts) r? `@ghost` `@rustbot` modify labels: rollup
Rollup of 13 pull requests Successful merges: - rust-lang/rust#148756 (Warn on codegen attributes on required trait methods) - rust-lang/rust#148790 (Add new Tier-3 target: riscv64im-unknown-none-elf) - rust-lang/rust#149271 (feat: dlopen Enzyme) - rust-lang/rust#149459 (std: sys: fs: uefi: Implement set_times and set_perm) - rust-lang/rust#149771 (bootstrap readme: make easy to read when editor wrapping is not enabled) - rust-lang/rust#149856 (Provide an extended framework for type visit, for use in rust-analyzer) - rust-lang/rust#149950 (Simplify how inline asm handles `MaybeUninit`) - rust-lang/rust#150014 (Metadata loader cleanups) - rust-lang/rust#150021 (document that mpmc channels deliver an item to (at most) one receiver) - rust-lang/rust#150029 (Update books) - rust-lang/rust#150031 (assert impossible branch is impossible) - rust-lang/rust#150034 (do not add `I-prioritize` when `F-*` labels are present) - rust-lang/rust#150036 (Use the embeddable filename for coverage artifacts) r? `@ghost` `@rustbot` modify labels: rollup


related issue: #145899
related pr: #146623
This PR is a continuation of #146623
I refactored some code for #146623 and added the functions shown in #144197
r? @bjorn3
cc: @ZuseZ4
Zulip link: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libload.20.2F.20dlopen.20Enzyme.2Fautodiff/near/553647912